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1 .1  Introduction 

The  Sensory  Communication  Group  is  conducting  research  on  (1)  the  auditory  and  tactual  senses,  (2) 
speech-reception  aids  (both  auditory  and  tactual)  for  individuals  who  are  hearing-impaired  or  deaf,  and 
(3)  human-machine  interfaces  for  teleoperator  and  virtual-environment  systems  (involving  the  visual  as 
well  as  the  auditory  and  tactual  senses).  Within  the  domain  of  hearing  aids,  research  is  being  conducted 
on  systems  that  bypass  the  outer  and  middle  ear  and  directly  stimulate  the  auditory  nerve  electrically 
(cochlear  prostheses),  as  well  as  on  systems  that  stimulate  the  system  acoustically.  The  research  on 
taction  is  focused  not  only  on  speech  reception  for  the  totally  deaf,  but  also  on  the  ability  of  the  human 
hand  to  sense  and  manipulate  the  environment.  Within  the  domain  of  human-machine  interfaces,  topics 
of  special  interest  concern  the  development  of  principles  for  mapping  the  human  sensorimotor  system  into 
non-anthropomorphic  slave  mechanisms  (or  the  equivalent  in  virtual  space)  and  the  ability  of  the  human 
sensorimotor  system  to  adapt  to  alterations  of  normal  sensorimotor  loops  caused  by  the  presence  of  the 
interface. 


1.2  Hearing  Aid  Research 

Sponsor 

National  Institutes  of  Health 
Grants  R01  DC00117 

Project  Staff 

Dr.  Joan  M.  Besing,  Professor  Louis  D.  Braida, 
Lorraine  A.  Delhorne,  Nathaniel  I.  Duriach,  Dr. 
Kenneth  W.  Grant,  Dr.  Christine  M.  Rankovic,  Dr. 
Charlotte  M.  Reed,  Dr.  Rosalie  M.  Uchanski,  Dr. 
Victor  W.  Zue,  Dr.  Patrick  M.  Zurek 


This  research  is  directed  toward  the  development 
of  improved  hearing  aids  for  people  suffering  from 
hearing  impairments  that  cannot  be  treated  med¬ 
ically.  Since  the  major  problem  for  most  people 
with  impaired  hearing  is  a  degraded  ability  to 
understand  speech,  and  since  medical  treatments 
are  available  for  abnormalities  of  the  outer  and 
middle  ear,  the  work  is  directed  towards  aids  that 
better  match  speech  signals  to  residual  auditory 
function  in  people  with  impairments  central  to  the 
middle  ear.  More  specifically,  the  research  is 
directed  at  improving  speech  reception  for  people 
with  sensorineural  impairments.  The  work  per¬ 
formed  during  the  past  year  can  be  divided  into 
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four  project  areas;  linear  amplification,  speech 
token  variability,  prediction  of  speech  intelligibility, 
and  aids  to  speechreading. 


1.2.1  Linear  Amplification 

This  research  has  focused  on  (1 )  theoretical  anal¬ 
ysis  of  suggested  prescriptive  methods  for  fitting 
frequency-gain  characteristics  to  individual  lis¬ 
teners,  (2)  investigation  of  the  reduction  in  speech 
intelligibility  associated  with  strong  high  frequency 
emphasis,’  and  (3)  the  use  of  adaptive  frequency- 
gain  characteristics  to  combat  the  effects  of  back¬ 
ground  interference  on.  target  speech  intelligibility.^ 
Current  research  on  adaptive  frequency-gain  char¬ 
acteristics  includes  both  experimental  intelligibility 
studies  to  determine  the  potential  benefits  of  adap¬ 
tive  systems  and  development  of  real-time  signal- 
processing  for  implementing  such  systems. 


1.2.2  Speech  Token  Variability 

Performance  in  intelligibility  tests  for  speech  seg¬ 
ments  depends  on  the  set  of  segments  to  be  iden¬ 
tified  and  the  characteristics  of  the  talker.  It  also 
depends,  however,  on  the  number  and  variability 
of  the  speech  tokens  chosen  to  represent  a  given 
segment.  (The  use  of  a  single  token  for  each 
segment  leads  to  the  use  of  artifacts  as  perceptual 
cues  and  results  in  artificially  high  performance.) 
In  order  to  predict  performance  on  such  tests,  it  is 
necessary  to  model  the  effects  of  token  variability. 
With  this  goal  in  mind,  we  are  (1)  measuring  the 
physical  variability  of  tokens, ^  (2)  determining  the 
perceptual  effects  of  this  variability  on  intelligibility 
for  both  normal  and  impaired  listeners,*  and  (3) 
developing  a  model  that  relates  the  perceptual 
effects  to  the  acoustic  characteristics  of  the  stimuli. 


1.2.3  Prediction  of  Speech 
Intelligibility 

This  project  involves  the  development  of  computa¬ 
tional  methods  for  predicting  the  intelligibility  of 
speech  subjected  to  a  waveform  degradation.  Our 
work  in  this  area  during  the  past  grant  year  has 
consisted  of  three  components:  (1)  completion  of 
preliminary  experiments  using  a  two-talker  data¬ 
base  (to  determine  the  feasibility  of  our  approach); 
(2)  establishing  the  amount  of  speech  data 
required  for  our  base  set  of  experiments  (based  on 
theoretical  considerations  for  information  transfer); 
and  (3)  completion  of  the  first  portion  of  these 
base  experiments. 


1.2.4  Aids  to  Speechreading 

Our  study  of  aids  to  speechreading  has  focused  on 
(1 )  developing  a  theoretical  understanding  of  the 
effects  of  speechreading  supplements  and  (2) 
evaluating  the  training  requirements  involved  in 
teaching  Cornett's  manual  Cued  Speech  system  to 
listeners  with  normal  hearing.  The  theoretical  work 
exploits  two  related  models  of  speech  segment 
confusions:  a  model  that  describes  confusions  in 
terms  of  a  multidimensional  Thurstonian  decision 
model^  and  a  model  that  specifies  the  relation 
between  the  decision  space  for  multimodal  (e.g., 
audiovisual)  presentation  conditions  and  the  con¬ 
stituent  unimodal  spaces.®  Our  results  on  the 
training  of  Cued  Speech  reception  suggest  that  the 
relatively  small  amount  of  training  required  to 
achieve  high  levels  of  performance  on  isolated 
speech  segments  is  a  poor  indicator  of  training 
requirements  for  sentence  reception. 


'  C.M.  Rankovic  and  P.M.  Zurek,  'Rollover  with  High-Frequency  Emphasis,"  J.  Acoust.  Soc.  Am.  87:  S87  (1990). 

2  C.M.  Rankovic,  R.L.  Freyman,  and  P.M.  Zurek,  "Potential  Benefits  of  Varying  the  Frequency-Gain  Characteristic  for 
Speech  Reception  in  Noise,"  submitted  to  J.  Acoust.  Soc.  Am. 

3  L.  Taylor,  Token  VariabiUtY  of  Intra-Speaker  Speech:  Fricative  Consonant  Sounds.  S.B.  thesis.  Dept,  of  Electr. 
Eng.  and  Comput.  Sci.,  MIT,  1990. 

♦  K.M.  Millier,  Intelligibility  of  Vowels  Represented  by  Multiple  Intra-Speaker  Tokens,  S.B.  thesis.  Dept,  of  Electr. 
Eng,  and  Comput.  Sci.,  MIT,  1990. 

s  L.D.  Braida,  'Development  of  a  Model  for  Multidimensional  Identification  Experiments,"  J.  Acoust.  Soc.  Am.  84; 
SI  42  (A)  1988. 

®  L.D.  Braida,  'Crossmodal  Integration  in  the  Identification  of  Consonant  Segments,"  Quart.  J.  Exper.  Psych.,  forth¬ 
coming;  L.D.  Braida,  "Two  Types  of  Audiovisual  Integration  tor  the  Identification  of  Speech  Segments,"  J. 
Acoust  Soc.  Am.  88;  S82  (1990). 
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1.3  Multiniicrophone  Hearing 
Aids 

Sponsor 

National  Institutes  of  Health 
Grant  2  ROi  DC00270 

Project  Staff 

Nathaniel  I.  Ourlach,  Dr.  Xiao  Dong  Pang,  Dr. 
William  M,  Rabinowitz,  Dr.  Patrick  M.  Zurek 

The  long-term  goal  of  this  research  is  the  develop¬ 
ment  of  sensory  aids  that  improve  the  ability  of 
hearing-impaired  listeners  to  function  in  complex 
acoustic  environments.  The  more  immediate  goal  is 
to  determine  the  benefits  that  can  be  achieved  for 
monaural  listening  through  the  use  of  microphone 
arrays  that  sample  the  acoustic  field  at  more  than 
one  point  in  space.  Since  the  reception  of  speech 
is  the  most  important  problem  for  the  hearing 
impaired,  the  target  signal  of  primary  interest  is 
speech. 

To  enhance  monaural  speech  reception,  we 
envision  a  microphone  array  that  resolves  the 
incoming  signals  into  simultaneous  directional 
channels,  followed  by  a  coding  operation  that 
transforms  these  resolved  signals  in  such  a  way 
that  resolution  is  preserved  at  the  perceptual  level 
after  the  signals  are  summed  for  presentation  to  a 
single  ear.’  Such  a  system  would  permit  the 
monaural  listener  (like  the  normal  binaural  listener) 
to  monitor  all  directions  simultaneously,  to  detect 
and  localize  in  the  same  operation,  and  to  focus 
on  a  single  direction.  Our  initial  work  on  the 
microphone  array  is  directed  toward  the  creation  of 
a  single  directional  channel  containing  the  target 
signal  (assumed  to  arise  from  a  target  source 
straight  ahead  of  the  listener)  and  the  reduction  of 
interference  from  sources  directionally  distinct  from 
the  target  source.  Parallel  processing  of  array 
outputs  to  achieve  simultaneous  multiple  direc¬ 
tional  channels  will  be  considered  only  after 
further  progress  on  the  coding  problem  has  been 
achieved. 


During  the  past  year,  work  has  focused  on 
acoustic  measurements  for  fixed-array  design, 
adaptive  beamforming  (analysis  of  an  adaptive 
noise  canceller  and  construction  of  a  real-time 
system),  and  experimental  studies  of  natural  spatial 
coding. 

1.3.1  Acoustic  Measurements  for 
Fixed-Array  Design 

We  have  completed  measurements  on  a  set  of 
transfer  functions  from  many  source  angles  within 
the  upper  hemisphere  to  10  microphone  locations 
situated  along  the  frame  of  eyeglasses  worn  by 
KEMAR  wihin  our  anechoic  chamber.  Two  arrays 
of  particular  interest  are  an  array  that  encompasses 
much  of  the  eyeglass  span  and  a  second  ‘'  n**  that 
is  substantially  smaller  (a  few  cm  span),  situated 
either  broadside  along  the  eyeglass  front  or  endfire 
along  the  temple.  We  have  also  begun  to  employ 
analytic  formulations  of  diffraction  and  scattering 
for  interpreting  the  acoustic  data  and  for  predicting 
the  performance  of  optimum  processors. 

1.3.2  Adaptive  Beamforming: 
Analysis  of  the  Adaptive  Noise 
Canceller 

The  adaptive  beamforming  algorithm  that  has  been 
the  focus  of  much  of  our  work,  the  Griffiths-Jim 
beamformer,8  can  be  viewed  as  a  simple  pre¬ 
processor  operating  in  conjunction  with  an  adap¬ 
tive  noise  canceller  (ANC),  which  was  described 
by  Widrow  et  al.®  Two  of  the  problems  encount¬ 
ered  with  this  beamforming  system  are  associated 
with,  and  can  be  described  more  simply  in  terms 
of,  the  ANC.  The  first  problem  is  known  as  misad- 
justment  and  is  due  to  fluctuations  in  the  adaptive 
filter's  weights.  The  second  problem  comes  from 
leakage  of  the  target  signal  into  the  reference 
channel,  which  is  ideally  free  of  target  for  best 
estimation  of  the  jammer.  Both  problems  increase 
in  severity  with  input  target-to-jammer  power 
ratio.  Our  previously- proposed  modifications  of  the 
Griffiths-Jim  beamformer  were  ad  hoc  attempts  to 
address  these  problems.’®  In  order  to  gain  a  more 


’  N.l.  Durlach,  R.C.  Corbett,  M.V.  McConnell,  W.M.  Rabinowitz,  P.M.  Peterson,  and  P.M.  Zurek,  "Multimicrophone 
Monaural  Hearing  Aids,"  RESNA  10th  Annual  Conference,  San  Jose,  California,  1987. 

*  LJ.  Griffiths  and  C.W.  Jim,  'An  Alternative  Approach  to  Linearly  Constrained  Adaptive  Beamforming,"  IEEE  Trans. 
Antennas  Propag.  AP-30:  27-34  (1982). 

»  B.  Widrow,  J.R.  Glover,  Jr.,  J.M.  McCool,  J.  Kaunitz,  C.S.  Williams,  R.H.  Hearn,  J.R.  Zeidler,  E.  Dong,  Jr.,  and 
R.C.  Goodlin,  'Adaptive  noise  cancelling;  Principles  and  Applications,"  Proc.  IEEE  63:  1692-1716  (1975). 

to  J.E.  Greenberg,  A  Real-time  Adaptive -beamforming  Hearing  Aid.  S.M.  thesis.  Dept,  of  Electr.  Eng.  and  Comput. 
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complete  understanding  of  the  limitations  of  the 
ANC,  we  are  now  attempting  to  derive  analytic 
expressions  for  system  performance. 


1.3.3  Adaptive  Beamforming:  A 
Head-Worn  Real-Time  System 

We  are  continuing  development  of  a  battery- 
powered,  body-worn  audio  processor  based  on  a 
Motorola  DSP56001  chip.  Functionally,  the  unit 
accepts  two  microphone  inputs  and  provides  a 
single  output.  The  hardware  is  general-purpose, 
and  the  device  can  be  programmed  to  perform  a 
wide  variey  of  algorithms  requiring  one  or  two 
input  signals.  To  date,  it  has  been  programmed  to 
implement  the  modified  Griffiths-Jim  beamformer 
as  well  as  a  simple  algorithm  that  automatically 
selects  the  microphone  signal  estimated  to  have 
less  noise  (due  to  head  shadow). 

1.3.4  Spatial  Coding:  Natural 
References 

Our  work  on  spatiai  coding'^  is  aimed  at  devel¬ 
oping  transformations  of  multiple  source  signals 
(which  we  expect  eventually  to  be  able  to  extract 
with  microphone  arrays)  to  render  these  signals 
maximally  intelligible  when  summed  for  monaural 
presentation.  As  background  for  this  work  we  have 
measured  the  extent  to  which  natural  coding 
systems— monaural  and  binaural  hearing— keep 
spatially-separate  sources  perceptually  separate. 
Specifically,  using  two  or  three  sources,  we  have 
measured  the  intelligibility  of  each  source  serving 
as  the  target  in  the  presence  of  the  others  as  inter¬ 
ference.  The  results,  averaged  over  source  config¬ 
urations,  show  an  advantage  of  about  10  dB  for 
binaural  over  monaural  listening  to  two  sources. 
As  expected,  this  binaural  advantage  is  reduced  by 
introducing  a  second  jammer. 


1 .4  Cochlear  Prostheses 

Sponsor 

National  Institutes  of  Health 
Grant  1  P01  DC0036112 

Project  Staff 

Professor  Louis  D.  Braida,  Lorraine  A.  Delhorne, 
Dr.  Donald  K.  Eddington,  Dr.  William  M. 
Rabinowitz,  Dr.  Charlotte  M.  Reed 

The  overall  goal  of  this  research  project  is  to  deter¬ 
mine  and  understand  the  potential  and  limitations 
of  cochlear  prostheses  and  to  develop  improved 
prostheses.  Residual  sensorineural  hearing  is 
measured  prior  to  implantation  in  connection  with 
subject  selection  and  to  provide  baseline  data  for 
later  comparison.  Following  implantation,  auditory 
performance  is  evaluated  by  studying  psychophys¬ 
ical  performance,  discrimination  of  speech  ele¬ 
ments,  and  comprehension  of  speech  and  the 
acoustic  environment.  Attempts  are  made  to  iden¬ 
tify  and  measure  central  processing  abilities  that 
are  relevant  to  speech-reception  performance  with 
an  implaiit  and  that  may  help  explain  the  large 
intersubject  variations  observed.  Also,  for  compar¬ 
ison  purposes,  speech-reception  tests  are  per¬ 
formed  using  a  promising  multichannel  tactile 
vocoder.  To  minimize  differential  learning  effects  in 
the  comparison,  these  tests  are  restricted  to  dis¬ 
crimination  of  speech  segments.  Further  research 
focuses  on  alternative  speech-processing  schemes 
to  achieve  improved  implant  performance.  This 
research  capitalizes  on  analytic  results  from  other 
parts  of  our  research  and  the  direct  accessibility  of 
the  implanted  electrode  array  (via  a  percutaneous 
plug).  During  the  past  year,  work  has  focused  on 
evaluation  of  overall  performance,  the  effects  of 
background  noise,  and  alternative  speech  pro¬ 
cessing. 


1.4.1  Overall  Performance 

This  work  is  concerned  with  documenting 
implantees'  speech-reception  abilities  at  a  variety 
of  different  levels  and  establishing  relationships 
among  different  tests.  At  present,  about  twenty 
subjects  have  participated  in  our  program  and 
about  eight  have  completed  most  of  our  routine 
evaluations.  Some  preliminary  observations  which 


Sci.,  MIT,  1989;  J.E.  Greenberg,  P.M.  Zurek,  and  P.M.  Peterson,  'Reducing  the  Effects  of  Target  Misalignment  in 
an  Adaptive  Beamformer  for  Hearing  Aids,'  J.  Acoust.  Soc.  Amer.  85:  S26  (1 989). 

”  C.R. Corbett,  Filtering  Competing  Messages  to  Enhance  Mutual  Intelligibility.  S.M.  thesis.  Dept,  of  Electr.  Eng.  and 
Comput.  Sci.,  MIT,  1986. 

Subcontract  from  Massachusetts  Eye  and  Ear  Infirmary.  Or.  Joseph  B.  Nadel,  M.D.,  Principal  Investigator. 
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appear  interesting  are  the  following:  Auditory  word 
recognition  with  the  lEEE/Harvard  sentences  cor¬ 
relates  highly  with  auditory  performance  on  the 
NU-6  monosyllabic  word  test  (R  =  0.98). 
Speech -segment  discrimination  tests  reveal  some 
subjects  with  relatively  good  performance  who  (on 
the  basis  of  other  tests)  derive  little  communicative 
benefit  from  their  implant;  these  subjects  may  be 
particularly  good  candidates  for  specialized  aural 
rehabilitation.  Finally,  measures  of  speech- 
segment  identification  performance,  in  contrast  to 
discrimination  performance,  relate  directly  to 
overall  measures  of  communicative  benefit. 


1.4.2  Effects  of  Background  Noise 

Everyday  communication  often  occurs  in  the  pres¬ 
ence  of  background  noise.  During  this  past  year 
we  began  studies  on  the  effects  of  background 
noise  on  speech  reception  with  the  Richards 
(Symbion)  implant  system.  In  comparison  to 
normal-hearing  subjects  tested  under  the  same 
conditions,  the  implantees  appear  about  15  dB 
more  sensitive  to  noise  for  both  consonant  and 
vowel  identification.  Analyses  are  presently 
underway  to  examine  the  dependencies  of  partic¬ 
ular  consonant  and  vowel  features  on  speech-to- 
noise  ratios. 


1.4.3  Alternative  Speech  and 
Processing 

Collaboration  with  Blake  Wilson  and  his  associates 
at  the  Research  Triangle  Institute  (RTI)  has  con¬ 
tinued.  Best  perceptual  results  have  been  obtained 
with  a  “supersampler"  scheme  consisting  of 
interleaved-pulse  stimulation  that  is  updated  over 
the  six  channels  at  a  rate  of  1600  Hz.  Substantial 
improvements  over  previous  results  (obtained  with 
the  normal  processor  which  employs  simultaneous 
multichannel  stimulation)  have  been  demonstrated 
for  both  monosyllabic  word  recognition  and  sen¬ 
tence  reception.^3 


1.5  Binaural  Hearing 

Sponsor 

National  Institutes  of  Health 
Grant  2  R01  DCOOIOO’* 

Project  Staff 

Nathaniel  I.  Durlach,  Dr.  Patrick  M.  Zurek 

The  long-term  goal  of  this  program  is  (1)  to 
develop  an  integrated,  quantitative  theory  of 
binaural  interaction  that  is  consistent  with  psycho¬ 
physical  and  physiological  data  on  normal  and 
impaired  auditory  systems  and  (2)  to  apply  our 
results  to  the  diagnosis  and  treatment  of  hearing 
impairments. 

Research  in  this  area  (performed  under  a  subcon¬ 
tract  from  Boston  University)  has  focused  on 
lateralization  phenomena.  One  series  of  studies  has 
explored  the  effect  of  onset  cues  for  long -duration 
sounds.  It  has  been  found  that  the  degree  to 
which  onset  information  dominates  lateralization 
depends  not  only  on  the  strength  and  consistency 
of  the  lateralization  information  in  the  ongoing 
stimulus,  but  also  on  the  extent  to  which  the 
ongoing  stimulus  appears  to  constitute  a  single 
"auditory  object."  Thus,  for  example,  the  onset's 
control  of  lateralization  can  be  greatly  reduced  by 
introducing  into  the  continuing  signal  an  abrupt 
change  in  amplitude  (an  increase  as  well  as  a 
decrease). 

Work  on  the  precedence  effect  with  pairs  of 
binaural  noise  bursts  has  shown  that  there  is  a 
large  variation  in  the  strength  of  the  effect  across 
noise  tokens  (some  tokens  even  show  a  reverse 
precedence  effect).  Efforts  are  now  being  made  ro 
determine  the  stimulus  variables  (such  as  phase 
spectrum,  cross-spectral  magnitude,  etc.)  that  are 
responsible  for  these  phenomena 

We  have  also  completed  an  analysis  of  the  proba¬ 
bility  distributions  of  interaural  cues  in  simple 
binaural  detection  stimuli.’®  This  report  presents 
derivations  of  these  distributions  in  calculable 
form,  as  well  as  a  summary  of  the  dependence  of 
variance  on  signal-to-noise  ratio. 


r®  B.S.  Wilson,  C.C.  Finley,  D.T.  Lawson,  R.D.  Wolford,  D.K.  Eddington,  and  W.M.  Rabinowitz,  "New  Levels  of 
Speech  Recognition  with  Cochlear  Implants,"  Nature,  forthcoming. 

Subcontract  from  Boston  University.  Professor  H.  Steven  Colburn,  Principal  Investigator. 

P.M.  Zurek,  ‘Probability  Distributions  of  Interaural  Phase  and  Level  Differences  in  Binaural  Detection  Stimuli," 
submitted  to  J.  Acoust  Soc.  Am. 
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1.6  Clinical  Applications  of 
Binaural  Hearing 

Sponsor 

National  Institutes  of  Health 
Grant  FV00428  '6 

Project  Staff 
Dr.  Patrick  M.  Zurek 

Work  on  clinical  applications  of  binaural  hearing 
has  focused  on  comparisons  between  the  perform¬ 
ance  of  listeners  with  sensorineural  impairments 
and  listeners  with  normal  hearing  on  tests  of 
binaural  detection,  localization,  and  contralateral 
masking.  The  results  show,  as  expected,  that 
impairments  often  lead  to  reduced  performance  in 
these  tests,  and  that  performance  is  correlated 
across  tests.  However,  correlation  between  per¬ 
formance  on  these  tests  and  audiometric  config¬ 
uration  is  relatively  weak;  unexplained  intersubject 
variance  remains  disturbingly  large. 

1.7  Tactile  Communication  of 
Speech 

Sponsor 

National  Institutes  of  Health 
Grant  5  R01  DC00126 

Project  Staff 

Lorraine  m.  DetriOiire,  i^oihoniei  1.  Lufidch,  Dr. 
William  M.  Rabinowitz,  Dr.  Charlotte  M.  Reed,  Dr. 
Mandayam  A.  Srinivasan 

Previous  research  on  (1)  tactual  communication  of 
speech  among  the  deaf-blind  in  which  the  tactual 
stimulation  is  achieved  by  direct  physical  contact 


with  the  speaker  and  (2)  tactual  aids  that  trans¬ 
form  acoustic  signals  into  patterns  of  tactual  stim¬ 
ulation  and  thus  function  at  a  distance  has  led  us 
to  the  following  conclusions:  (1 )  The  tactual  sense 
is  capable  of  receiving  continuous  speech  at  nearly 
normal  speaking  rates  with  nearly  zero  error  rates; 
(2)  Subjects  are  capable  of  integrating  a  relatively 
impoverished  tactual  signal  with  visual 
speechreading  to  achieve  essentially  normal 
speech-reception  performance;  (3)  Limitations  on 
the  speech -reception  performance  obtained  with 
current  tactual  aids  are  due  primarily  to  inadequa¬ 
cies  in  the  design  of  the  aids  and/or  in  the  training 
received  with  these  aids;  and  (4)  There  are  no  fun¬ 
damental  scientific  obstacles  to  eliminating  these 
inadequacies  and  achieving  much  improved 
speech  reception  for  a  wide  range  of  patients. 
Conclusions  (1)  and  (2)  are  based  on  (a)  extrapo¬ 
lations  of  results  obtained  with  relatively  crude 
experimental  devices  on  subjects  with  relatively 
limited  training;’’  (b)  tactual  communication  per¬ 
formance  exhibited  by  the  deaf-blind  using  direct- 
contact  methods;’®  and  (c)  preliminary  results  on 
the  ability  of  individuals  who  are  experienced  in 
the  visual  reception  of  Cued  Speech  to  integrate 
visual  speechreading  with  direct-contact  tactual 
cueing.  Conclusions  (3)  and  (4)  are  essentially 
corollaries  of  conclusions  (1)  and  (2). 

With  the  above  lesearch  as  background,  our  ulti¬ 
mate  general  goal  can  be  stated  simply  as  follows: 
To  develop  schemes  for  processing  acoustic 
signals  and  displaying  these  processed  signals  to 
the  tactual  sense  so  that,  when  accompanied  by 
equivalent  training,  they  provide  speech -reception 
performance  comparable  to  that  demonstrated  for 
me  oirect-contact  methods.  Discussion  of  our  pro¬ 
gress  during  the  past  year  is  divided  into  the  fol¬ 
lowing  subsections:  (a)  Basic  study  of  encoding 
and  display  schemes,  (b)  Tactual  supplements  to 
speechreading,  (c)  Evaluation  of  practical  aids, 
and  (d)  Completion  of  work  on  natural  methods  of 


’«  Subcontract  from  University  of  Connecticut.  Dr.  Janet  D.  Koehnke,  Principal  Investigator. 

”  N.l.  Durlach,  C.E.  Sherrick,  and  J.D.  Miller,  ‘Sensory  Substitution:  Visual  and  Tactual  Methods,'  chapter  in 
Speech  Communication  Aids  for  the  Hearing  Impaired:  Current  Status  and  Needed  Research.  Report  of  CHABA 
Working  Group  95  by  C.S.  Watson,  R.A.  Dobie,  N.l.  Durlach,  H.  Levitt,  J.D.  Miller,  C.E.  Sherrick,  F.B.  Simmons, 
G.A.  Studebaker,  R.S.  Tyler,  and  G.P.  Widin,  forthcoming;  C.M.  Reed,  N.l.  Durlach,  and  L.D.  Braida,  “Research  on 
Tactile  Communication  of  Speech:  A  Review,'  ASHA  Monograph  Number  20,  1982;  C.E.  Sherrick,  ‘Basic  and 
Applied  Research  on  Tactile  Aids  for  Deaf  People:  Progress  and  Prospects,'  J.  Acoust.  Soc.  Am.  75:  1325-1342 
(1984). 

’«  C.M.  Reed,  W.M.  Rabinowitz,  N.l.  Durlach,  L.D.  Braida,  S.  Conway- Fithian,  and  M.C.  Schultz,  ‘Research  on  the 
Tadoma  Method  of  Speech  Communication,'  J.  Acoust.  Soc.  Am.  77:  247-257  (1985). 
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tactual  communication.  Recent  reports  of  our  work 
are  available  in  Reed  et  at.;^^  and  Tan  et  al.2° 

1.7.1  Basic  Study  of  Encoding  and 
Display  Schemes 

Previous  research  on  transmitting  information 
through  the  tactual  (i.e.,  cutaneous,  proprio¬ 
ceptive,  kinesthetic)  sense  has  been  confined 
almost  exclusively  to  stimulation  of  the  skin; 
manipulation  of  joint  angles  (i.e.,  hand  postures) 
has  been  essentially  ignored.  Research  is  now 
underway  to  determine  information  transfer  rates 
achieved  by  manipulation  of  joint  angles,  as  well 
as  by  both  joint  angle  manipulation  and  cutaneous 
stimulation  combined.  New  devices  to  drive  the 
hand  are  being  developed  and  preliminary  discrim¬ 
ination  and  identification  experiments  are  being 
performed.  One  issue  of  particular  interest  con¬ 
cerns  the  extent  to  which  subjects  highly  experi¬ 
enced  in  outputting  via  manual  manipulation 
(such  as  typists  and  pianists)  are  exceptionally 
good  at  learning  to  receiving  information  manually. 

1.7.2  Tactual  Supplements  to 
Speechreading 

Research  in  this  area  over  the  past  year  has  been 
concerned  with  comparing  the  effectiveness  of  a 
simple,  low-bandwidth  supplement  to 
speechreading  presented  auditorally  versus 
tactually  The  signal  consisted  of  a  200- Hz  tone 
modulated  by  the  envelope  of  an  octave  band  of 
speech  centered  at  500  Hz.  The  resulting  signal 
was  used  to  drive  a  high-performance  single¬ 
channel  vibrator  for  tactile  stimulation  or  Koss 
earphones  for  aud'tory  stimulation.  Preliminary 
results  indicate  a  consistent  improvement  in 
speechreading  using  the  single-channel  vibrator; 
however,  this  benefit  is  small  compared  to  that 
obtained  with  the  auditory  supplement.  Various 


hypotheses  concerning  the  observed  difference  in 
performance  between  the  tactile  and  auditory  sup¬ 
plements  are  now  being  examined. 


1.7.3  Evaluation  of  Practical  Aids 

Experiments  are  being  conducted  to  compare  seg¬ 
mental  resolution  through  a  wearable  tactile  device 
(the  seven-channel  Tactaid  VII  which  uses  zero¬ 
crossing  analysis  to  track  the  first  two  speech 
formants)  and  a  laboratory  version  of  the  Queen  s 
University  tactile  aid  (a  nine-channel  device  which 
operates  using  a  traditional  vocoder  principle). 
Results  currently  available  indicate  roughly  similar 
performance  through  the  two  devices.  An  initial 
evaluation  of  a  profoundly  hearing-impaired  adult 
(who  had  worn  a  Tactaid  VII  for  approximately 
two  months  prior  to  testing)  indicate  substantial 
benefits  to  speechreading  for  segments  but  only 
modest  benefits  for  sentence  materials.  During  the 
coming  year,  we  plan  to  re-test  this  subject  as  well 
as  to  provide  Tactaid  VII  devices  to  additional  sub 
jects  who  will  participate  in  periodic  evaluations. 

1.7.4  Completion  of  Work  on 
Natural  Methods  of  Tactual 
Communication 

During  the  past  year,  we  have  continued  work  on 
our  detailed  statistical  analysis  of  measurements  of 
speech  produced  by  Tadoma  users  (in  Tadoma, 
speech  is  understood  by  using  the  hand  to  sense 
the  mechanical  actions  of  the  face  associated  with 
speech  production);  data  collection  and  analysis 
for  a  study  of  the  visual  reception  of  sign  language 
as  a  function  of  presentation  rate  to  serve  as  a  ref¬ 
erence  for  previous  results  on  the  tactual  reception 
of  sign  language;  and  further  studies  of  the  tactual 
reception  of  Cued  Speech  in  conjunction  with 
both  speechreading  and  Tadoma. 


’9  C.M.  Reed,  N.l.  Durlach,  L.D.  Braida,  and  M.C.  Schultz,  'Analytic  Study  of  the  Tadoma  Method:  Effects  of  Hand 
Position  on  Segmental  Speech  Perception,"  J.  Speech  Hear.  Res.  32.  921-929  (1989);  C.M.  Reed,  N.l.  Durlach, 
L  A.  Delhorne,  W.M.  Rabinowitz,  and  K.W.  Grant,  'Research  on  Tactual  Communication  of  Speech;  Idea,  Issues, 
and  Findings,"  Volta  Rev.  91;  65-78  (1989);  Also  in  Research  on  the  Use  of  Sensory  Aids  for  Hearing -Impaired 
People,  ed.  N.S.  McGarr;  C.M.  Reed,  L.A.  Delhorne,  N.l.  Durlach,  and  S.D.  Fischer,  "A  Study  of  the  Tactual  and 
Visual  Reception  of  Fingerspelling,' j.  Speech  Hear.  Res.  33.  786-797  (1990);  C  M.  Reed,  W.M.  Rabinowitz,  N.l 
Durlach,  L.A.  Delhorne,  L.D.  Braida,  J.C.  Pemberton,  B.D.  Mulcahey,  and  D.L.  Washington,  "Analytic  Study  of  the 
Tadoma  Method;  Improving  Performance  through  the  Use  of  Supplementary  Tactual  Displays,"  submitted  to  J. 
Speech  Hear.  Res.,  C  M.  Reed,  N.l.  Durlach,  and  L.A.  Delhorne,  'Natural  Methods  of  Tactual  Communication," 
chapter  in  Tactile  Aids  for  the  Hearing  Impaired,  ed.  Ian  R.  Summers  (New  York;  Taylor  and  Francis  Ltd.,  forth¬ 
coming). 

M  H.7..  Tan,  W.M.  Rabincw'tz,  and  N.l.  Durlach,  'Analysis  of  a  Synthetic  Tadoma  System  as  a  Multidimensional 
Tactile  Display,"  J.  Acoust.  Soc.  Am.  86;  981  -988  0989). 
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1.8  Super  A' Jitory  Localization 
for  Improved  Human-Machine 
Interfaces 

Sponsor 

(J.S.  Air  Force  -  Office  of  Scientific  Research 
Grant  AFOSR  90-200 

Project  Staff 

Nathaniel  I.  Durlach,  Dr.  Richard  M.  Held.  Dr.  Xiao 
Dong  Pang,  Dr.  William  M.  Rabinowitz,  Dr.  Patrick 
M.  Zurek 

The  normal  human  auditory  system  suffers  from  a 
number  of  deficiencies  in  its  ability  to  localize 
sound  sources.  The  auditory  system  determines  the 
distance  or  elevation  of  a  source  poorly,  is  sub¬ 
stantially  worse  at  detecting  changes  in  azimuth 
when  the  source  is  off  to  the  side  than  when  it  is 
in  front  or  back,  and  occasionally  makes  front- 
back  confusions  when  the  head  is  motionless. 
However,  when  localization  is  considered  in  the 
context  of  human-machine  interfaces  such  as 
those  employed  in  teleoperator  or  virtual- 
environment  systems,  there  is  an  opponunity  to 
recode  source  location  in  a  manner  that  improves 
localization  In  other  words,  one  can  transform  the 
acoustical  cues  available  to  the  listener  for 
determing  source  location  (i.e.,  alter  the  manner  in 
which  source  location  is  represented  in  the 
binaural  acoustic  stimulus)  in  such  a  way  that  the 
listener  achieves  super  localization. 

The  principal  research  questions  in  attempting  to 
achieve  super  localization  concern  the  ability  of 
the  human  operator  to  adapt  to  such  transforma¬ 
tions  and  to  switch  back  and  forth  between  the 
normal  cue  system  and  the  altered  cue  system 
rapidly  and  reliably.  Although  certain  conse¬ 
quences  of  these  transformations  can  be  predicted 
theoretically  from  our  models  of  normal  human 
audition,  those  involving  perceptual  learning  and 
adaotation  cannot  be  predicted.  A  major  goal  of 
the  research  is  to  determine,  understand,  and 
model  the  perceptual  effects  of  these  transforma¬ 
tions. 

The  planned  research  involves  the  study  of  adapta¬ 
tion  to  a  wide  range  of  transformations  using  a 
specially  designed  virtual-environment  system  for 
presenting  the  transformed  localization  cues,  a 
variety  of  training  procedures  to  achieve  adapta¬ 
tion,  and  localization  tests  to  measure  adaptation 
that  include  detection  and  localization  in  multiple- 


source  environments  and  dynamic  tests  of  localiza¬ 
tion  constancy,  as  well  as  discrimination  and 
identification  of  single  sources.  To  the  extent  that 
reliable  and  reversible  adaptation  can  be  demon¬ 
strated,  the  results  of  this  research  will  provide 
important  new  options  for  improved  interface 
design.  Further  background  in  this  area  can  be 
found  in  Durlach. 

During  the  past  year,  our  efforts  have  been 
directed  primarily  towards  the  development  of 
appropriate  facilities  for  this  work. 

Our  desire  to  incorporate  reverbe'^ation  into  head- 
related  transfer  functions  (HHTFs)  for  virtual 
auditory  displays  has  resulted  in  some  suggested 
changes  in  the  Convolvotron,  a  key  device  for  cre¬ 
ating  virtual  auditory  environments.  Numerous  dis¬ 
cussions  between  our  staff  and  the  manufacturer 
of  the  Convolvotron  have  lead  to  an  outline  for 
improvement  of  the  filter  length,  memory  size, 
computational  method,  speed,  and  data  structure 
of  the  system.  Also,  at  our  request,  the  manufac¬ 
turer  has  developed  a  new  software  package  for 
making  HRTF  measurements  in  a  reverberant  envi¬ 
ronment  using  the  Convolvotron.  The  modified 
system  is  currently  under  evaluation  in  our  labora¬ 
tory. 

In  order  to  be  able  to  measure  HRTFs  for  indi¬ 
vidual  listeners,  a  computer-controlled  speaker 
placement  system  is  being  constructed  with  a 
speaker  mounted  on  an  8-ft.  diameter  ring  which 
can  be  driven  in  two  degrees  of  freedom  for 
placing  the  speaker  anywhere  on  the  surface  of  a 
sphere  whose  center  coincides  with  the  center  of 
subjects  s  head.  The  mechanical  structure  of  the 
system  has  been  completed,  including  the  ring,  the 
platform,  the  motors,  the  driving  mechanism,  and 
the  sensing  mechanism  for  motor  control.  A 
master's  thesis  has  been  outlined  to  implement  the 
remaining  control  mecnanisms,  measure  HRTFs, 
incorporate  the  HRTFs  into  the  Convolvotron,  and 
conduct  and  analyze  the  relevant  experiments. 

A  mechanical  head-tracker  with  six  degrees  of 
freedom  has  been  designed  and  is  under  con¬ 
struction  in  the  laboratory.  The  purpose  of  this 
tracking  device  is  to  overcome  three  major  prob¬ 
lems  with  the  currently  available  (Polhemus) 
tracker;  restricted  work  space,  time  delay,  and 
electromagnetic  interference.  The  mechanical 
tracker  will  provide  a  much  larger  operating  range 
than  the  Polhemus  tracker  (9-ft.  radius  as 
opposed  to  2-ft.),  will  be  essentially  immune  to 
electromagnetic  interference,  and  will  have  delays 


N.l.  Durlach,  'Auditory  Localization  in  Teleoperator  and  Virtual- Environment  Systems:  Ideas,  Issues,  and 
Problems,"  Percept.,  forthcoming. 


312  RLE  Progress  Repon  N';mber  133 


Chapter  1.  Sensory  Communication 


less  than  100  microseconds  (from  a  25  kHz  A/D 
convertor).  The  construction  of  the  mechanical 
head -tracker  and  its  interface  with  a  PC  are 
expected  to  be  completed  within  the  next  few 
months. 

A  complete  software  system  for  conducting  local¬ 
ization  and  adaptation  experiments  has  been 
developed.  This  system  includes  a  menu-driven 
user  interface  for  entering  and  displaying  exper¬ 
imental  parameters,  and  a  “controller'  for  cali¬ 
brating  spatial  coordinate  systems,  controlling 
stimulus  presentations  (both  probabilistically  and 
temporally),  and  prompting,  recording,  storing,  and 
organizing  subject  responses.  The  initial  transfor¬ 
mations  implemented  include  rotation  of  the 
intoraural  axis,  expansion  and  contraction  of  the 
interaural  axis,  and  exponentiation  of  HRTFs.^^  A 
computer-controlled  visual  display  is  available  in 
the  form  of  spatially  distributed  light  bulbs.  In 
addition  to  conventional  keyboard  control,  manual 
control  of  sound  source  location  has  been  maoe 
available  through  the  use  of  a  Power  Glove.  Pre¬ 
liminary  observations  suggest  that  the  use  of  these 
non-auditory  inputs  improves  externalization  of 
virtual  acoustic  targets. 

A  DEC  5000  work  station  with  a  graphics  engine 
has  been  purchased  and  a  graduate  student  has 
begun  to  develop  software  for  generating  3-D 
visual  stimuli  for  a  head-mounted  visual  display 
(to  be  integrated  with  the  auditory  display). 

A  preliminary  analysis  has  been  made  of  algo¬ 
rithms  for  achieving  interaural  expansion.  The 
algorithms  considered  are  those  of  Durlach  and 
Pang22  3pcl  Van  Veen  and  Jenison.^^  The  second 
set,  unlike  the  first,  are  linear.  Thus,  they  should  be 
superior  in  multiple  jammer  environments  (to  the 
extent  that  the  approximations  are  accepUlile).  For 
virtual  auditory  environments,  however,  where  the 
experimenter  has  separate  control  of  individual 
sound  sources,  issues  other  than  linearity  domi¬ 
nate. 


1.9  Research  on 
Reduced-Capability  Human 
Hands 

Sponsor 

U.S  Navy  -  Office  of  Naval  Research 
Grant  N00014-90-J-1935 

Project  Staff 

Lorraine  A.  Delhorne,  Nathaniel  I.  Durlach,  Dr. 
Xiao  Dong  Pang,  Dr.  Mandayam  A.  Srinivasan 

The  general  objectives  of  our  research  on  hand 
function  are  to  increase  basic  knowledge  of 
manual  sensing  and  manipulation,  aid  in  the 
design  and  evaluation  of  artificial  hands  for  robotic 
and  teleoperator  systems,  and  improve  clinical 
diagnosis  and  treatment  of  hand  impairments. 
Reports  of  some  of  our  previous  work  in  this 
general  area  are  available  in  Durlach  et  al.^'*  and 
Pang  et  al.^® 

The  particular  research  being  conducted  in  this 
grant  focuses  on  the  ability  of  the  human  hand  to 
sense  and  manipulate  the  environment  under 
various  types  of  constraints  Such  constrained 
hand  performance  is  being  studied  in  connection 
with  hand-design  questions  in  the  area  of  teleop¬ 
erator  systems.  Determination  of  how  performance 
degrades  as  various  capabilities  of  the  normal 
human  hand  are  elimiriated  provides  important 
background  for  hand-design  decisions.  The  con¬ 
straints  to  be  studied  are  imposed  by  experimental 
gloves  and  local  anesthetics,  as  well  as  by  various 
types  of  hand  impairments  (resulting  from  injuries, 
birth  defects,  or  diseases). 

Efforts  in  the  first  area  have  involved  (1)  inter¬ 
action  with  clinics  to  establish  a  flow  of  patients 
with  appropriately  impaired  hands,  (2)  study  of 
techniques  for  constructing  experimental  gloves 
that  reduce  tactile  sensitivity  in  a  controliable 
manner,  and  (3)  selection  and  acquisition  of  mate¬ 
rials  for  constructing  various  types  of  splints  for 
limiting  movement  and  force  output  of  the  hand. 
The  technique  for  glove  construction  that  appears 
most  promising  involves  the  dipping  of  latex  finger 
cots  in  plastic  coating  material. 


22  N.l  Durlach  and  X.D.  Pang,  “Interaural  Magnification,'  J.  Acoust.  Soc.  Am.  80:  1849-1850  (1986). 

23  B.  Van  Veen  and  R.  Jenison,  “Auditory  Space  Expansion  via  Linear  Filtering,"  J.  Acoust.  Soc.  Am.,  forthcoming. 

2<  N.l.  Durlach,  L.A.  Delhorne,  A.  Wong,  W.Y.  Ko,  W.M.  Rabinowitz,  and  J.M.  Hollerbach,  "Manual  Discrimination 
and  Identification  of  Length  by  the  Finger-Span  Method,"  Percept.  Psychophys.  46(1):  29-38  (1989). 

25  X.D.  Pang  H.Z.  Tan,  and  N.l.  Durlach,  'Manual  Discrimination  of  Force  Using  Active  Finger  Motion,"  Percept. 
Psychophys.,  forthcoming. 
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Efforts  in  the  second  area  have  involved  (1)  con¬ 
ceptual  analysis  of  how  various  types  of  manual 
tasks/tests  relate  to  various  underlying  sensimotor 
capabilities  of  the  hand;  (2)  selection  of  specific 
tasks/  tests  in  the  clinical  domain,  the  elementary 
analytic  domain,  the  general  functional  domain, 
and  the  teleoperator  operational  domain;  and  (3) 
development  of  facilities  for  performing  these 
tasks/te'ts  and  for  recording  hand  actions  during 
the  tasks/tests. 


1.10  Skin  Biomechanirs 

Sponsor 

National  Institutes  of  Health 
Grant  5  R29  DC00625 

Project  Staff 

Dr.  Mandayam  A.  Srinivasan 

Whenever  we  touch  an  object,  the  source  of  all 
tactile  information  is  the  spatio-temporal  distribu¬ 
tion  of  mechanical  loads  on  the  skin  at  the  contact 
interface.  The  relationship  between  these  loads 
and  the  resulting  stresses  and  strains  at  the  nerve 
terminals  within  the  skin  plays  a  fundamental  role 
in  the  neural  coding  of  tactile  information. 
Although  empirical  determination  of  the  stress  or 
strain  state  of  a  mechanoreceptor  is  not  possible  at 
present,  mechanistic  models  of  the  skin  and 
subcutaneous  tissues  enable  prediction  and  verifi¬ 
cation  of  peripheral  neural  response.  The  research 
under  this  grant  is  directed  towards  applying  the¬ 
oretical  and  computational  mechanics  to  analyze 
the  biomechanical  aspects  of  touch— the 
mechanics  of  contact,  the  transmission  of  the 
mechanical  signals  through  the  skin,  and  their 
transduction  into  neural  impulses  by  the 
mechanoreceptors. 


1.10.1  Determination  of  Geometric 
and  Material  Properties  of  the 
Primate  Fingertip 

The  first  step  in  performing  mechanistic  analyses 
of  the  primate  fingertip  is  to  determine  its 
geometric  and  material  properties.  We  have 
indented  the  fingerpads  of  humans  and  monkeys 
in  vivo  using  a  line  load  delivered  by  a  sharp 
wedge,  and  photographed  the  resulting  skin 


surface  deflections.  We  have  shown  that  the 
homogeneous  elastic  model  of  the  fingertip  only 
roughly  approximates  the  experimental  data,  while 
a  si'^ple  alternative  model,  which  views  the 
fingertip  as  an  elastic  membrane  filled  with  an 
incompressible  fluid  (like  a  "waterbed"),  predicted 
the  observed  profiles  very  well.^s  We  are  planning 
more  experiments  to  determine  the  external  and 
internal  geometry  of  the  primate  fingertip  as  well 
as  its  material  properties. 

1.10.2  Finite  Element  Analyses  of 
Two-dimensional  Models 

We  have  performed  finite  element  analysis  on  a 
series  of  mechanistic  models  of  the  fingerpad 
under  a  variety  of  mechanical  stimuli.  The  models 
range  from  a  semi-infinite  medium  to  cylindrical 
distal  phalanx,  composed  of  either  a  homogeneous 
elastic  material  or  a  thick  elastic  shell  containing  a 
fluid.  Simulations  of  the  mechanistic  aspects  of 
neurophysiological  experiments  involving  mapping 
of  receptive  fields  with  single  point  loads,  determi¬ 
nation  of  spatial  resolution  of  two-point  stimuli, 
and  indentations  by  single  bars  as  well  as  periodic 
and  aperiodic  gratings  have  been  carried  out.  The 
results  show,  for  example,  that  the  strain  energy 
density  at  the  receptor  site  is  a  leading  contender 
for  the  relevant  stimulus  that  causes  the  responses 
recorded  from  slowly  adapting  afferent  fibers. 
More  generally,  we  have  demonstrated  the  power 
of  computational  mechanics  in  investigating  the 
relationship  between  tactile  stimuli  imposed  on  the 
skin  and  the  resulting  peripheral  neural  response. 
The  analyses  are  being  extended  both  in  terms  of 
the  mechanical  stimuli  applied  on  the  fingerpad  as 
well  as  the  geometrical  and  material  properties  of 
the  fingertip. 

1.10.3  Tactile  Sensing  of  Shapes 
and  Softness 

We  have  been  collaborating  with  Dr.  LaMotte  of 
Yale  University  School  of  Medicine  in  conducting 
psychophysical  and  neurophysiological  studies  of 
the  encoding  of  shapes  and  degree  of  softness  in 
mechanoreceptive  afferents.  Based  on  a  theoretical 
analysis  of  the  mechanics  of  contact,  we  have 
demonstrated  that  the  receptors  respond  to  the 
low- pass  filtered  versions  of  surface  pressures.^’’ 
Thus,  curvature  of  the  skin  surface  under  an 


26  M.A.  Srinivasan,  “Surface  Deflection  of  Primate  Fingertip  Under  Line  Load,"  J.  Biomech.  22(4):  343-349  (1989). 

27  M.A.  Srinivasan  and  R.H.  LaMotte,  “Encoding  of  Shape  in  the  Responses  of  Cutaneous  Mechanoreceptors,"  in 
Information  Processing  in  the  Somatosensory  System,  eds:  0.  Franzen  and  J.  Westman,  Wenner-Gren  Interna- 
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object,  which  we  know  from,  different’^!  geometry 
is  approximated  by  the  second  spatial  derivative  of 
surface  deflection,  is  coded  without  differentiating 
(which  is  a  noise  enhancing  process),  but  by 
exploiting  its  relation  to  surface  pressure.  We  will 
be  using  our  finite  element  models  to  further 
explore  the  neural  coding  of  shapes  and  softness. 

1.10.4  Development  of  a 
Computational  Theory  of  Touch 

Although  the  'hardware'  of  the  tactile  apparatus  in 
humans  and  robots  is  different,  they  have  the 
common  feature  of  mechanosensors  embedded  in 
a  deformable  medium.  Thus  the  computational 
problem  of  coding  (predicting  sensor  response  for 
a  given  mechanical  stimulus  at  the  surface)  and 
decoding  (inferring  the  mechanical  stimulus  at  the 
surface  by  suitably  processing  the  sensor 
response)  need  similar  mechanistic  analyses  for 
their  solution.  We  have  developed  such  a  "compu¬ 
tational  theory"  for  an  idealized  medium  subjected 
to  arbitrary  pressure  or  displacement  loading  con¬ 
ditions,  and  give  explicit  formulae  for  the  coding 
and  decoding  problems.^s 

In  collaboration  with  Dr.  Annaswamy  of  the 
Department  of  Aerospace  and  Mechanical  Engi¬ 
neering,  Boston  University,  we  have  investigated 
some  of  the  identification  and  control  problems 
that  occur  in  the  context  of  manipulation,  when 
compliance  is  present  in  the  end-effectors  as  well 
as  in  the  object. In  order  to  understand  the  fun¬ 
damental  aspects  of  these  tasks,  we  have  analyzed 
the  problem  of  identification  of  compliant  objects 
with  a  single  finger  contact,  was  well  as  under  a 
two-finger  grasp.  Assuming  that  the  finger  and  the 
compliant  object  are  constrained  to  deform  along  a 
single  spatial  dimens'on,  we  have  carried  out 
parameter  identification  using  either  force  or  dis¬ 
placement  inputs  to  the  rigid  backing  of  the  end- 
effector.  Based  on  this  analysis,  control  strategies 
are  developed  to  achieve  a  desired  manipulation  of 
the  object  in  the  workspace.  Animated  graphical 
renderings  are  being  developed  to  visually  illus¬ 
trate  the  presence  of  absence  of  slipping  and 


crushing  during  an  active  manipulation  task.  The 
theoretical  results  can  be  used  tc  generate  testable 
hypotheses  for  human  or  robot  experiments  on 
tactual  sense. 
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